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ABSTRACT 

The traditional model used by all institutions of 
higher education include the three broad areas of teaching, 
scholarship, and service as part of faculty evaluation. This paper 
examines the definition of scholarship in higher education, focusing 
on current definitions of scholarship, administrators' perceptions 
about scholarship, measures of scholarship, and emerging trends in 
recommendations about scholarship. The tr?ditional model of 
scholarship ranked teaching ahead of research and service, unlike 
recent practice with ills emphasis on research *nd publication. The 
majority of faculty in the social sciences, however, believes in the 
importance of research and publication in scholarship, but feel that 
too much emphasis is placed on publication. Administrators have 
generally approved of research and publication as a method of 
evaluating contributions made by faculty. Measures of scholarship in 
recent years have centered on the quantity of articles and monographs 
produced by faculty members. Recent trends in the definition of 
scholarship have centered on the need to move beyond publication 
counts in the evaluation of faculty. Some researchers suggest that 
new definitions of scholarship should place more emphasis on teaching 
and service. (Contains 67 references.) (MDM) 
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A REVIEW OF THE CONSTRUCT OF SCHOLARSHIP IN THE LITERATURE 



Abstract 



The traditional model used by all institutions of higher education include the three broad 
areas of teaching, scholarship, and service as part of faculty evaluation. The author reviews 
the literature to explore the role scholarship plays in faculty evaluation. Five major 
categories relative to scholarship were identified: current definitions, faculty perceptions, 
administrators' perceptions, measures of scholarship, and emerging trends in 
recommendations about scholarship. It was within this context that this review was 
undertaken and situated. 



A REVIEW OF THE CONSTRUCT OF SCHOLARSHIP IN THE LITERATURE 

Introduction 

The traditional model used by all institutions of higher education, regardless 
of discipline, include the thiee broad areas of teaching, research, and service as part 
of faculty evaluation (Braxton & Toombs, 1980; Crosson, 1983; Seldin, 1984). In 
earlier times, the element of teaching was considered to be the most important, 
followed by service, then research (Boyer, 1990). This traditional model continues 
in use today and there is widespread agreement over its use regardless of institutional 
type, purpose, or discipline. However, there has been a shift in the importance 
assigned to each element of the model based on institutional type, purpose, and 
discipline. 

Over the past twenty years, the new ordering of the model's elements in 
doctoral granting institutions in almost all disciplines ranks research as the most 
important criteria, teaching as second, and service as least important (Blake & 
Tjoumas, 1990; McShane & Douzenis, 1987; Seldin, 1984). According to the 
literature, there are no studies which provide evidence on the ranking of the model's 
criteria for professional schools of education. However, the following five major 
categories relative to scho'arship were identified in the literature: current definitions 
of scholarship, faculty perceptions about scholarship, administrators' perceptions 
about scholarship, measures of scholarship, and emerging trends in recommendations 
about scholarship. It was within this context that the following review of the literature 
was undertaken and situated. 



Current Definitions of Scholarship 
Traditionally the term "scholarship" has come to mean being involved in 
research and publication (Boyer, 1990; Miller & Serzan, 1984; Sundre, 1990; West, 
Hore, & Boon, 1980). It has tended to be narrowly defined as systematic inquiry 
leading to products such as publication of books and articles in professional journals 
(Blackburn, Bieber, Lawrence, & Trautvetter, 1991; Pellino, Blackburn, & Boberg, 
1984). This concept of scholarship, in and of itself, is not a problem. Nor is the 
need for research and publication in question. The problem is that neither all 
institutions nor ali members of faculty within these institutions agree and/or support 
the current narrow definition of scholarship/publication. As such, the standard which 
is created by this narrow definition and used to evaluate faculty contributions is 
inherently inappropriate for these institutions and faculties. 

Miller (1984) asserts that there should be no agreement on categories of 
scholarship/publication. He favors the use of a definition which is dependent upon 
the nature of the institution which uses it. For example, how two-year colleges define 
scholarship and publication depends on their missions. These missions are different, 
not better or worse, but different from those of research institutions. 

Centra (1989) agreed with the assertion made by Miller (1984) by stating that 
the definition of scholarship depends on institutional type and purpose. However, 
Centra expands upon Miller's assertion by stating that acceptable performance in 
terms of scholarship varies, not only by institution and institutional purpose, but also 
by discipline. For example, he reports in a survey he conducted in 1977 involving 
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department chairs, that they indicated that journal articles and grants were critical to 
the evaluation of scholarship of faculty members in the natural sciences. 

Despite what one thinks, the current narrow definitions of scholarship and 
scholarly contributions are a recent phenomenon. In earlier times, scholarship was 
a broader concept than it is now (Boyer, 1990). According to Boyer (1990), 
scholarship referred to a variety of creative works the integrity of which was 
measured by the ability to think, communicate, and learn. In fact, Cardozier (1991) 
suggests that in many institutions the lack of research and publications had no 
influence on promotion and tenure decisions until after the 1960 ! s and Sputnik. This 
was validated by Beiber and Blackburn (1989). However, what we now have in 
academe is a departure from scholarship's earlier, more comprehensive meaning. We 
have a more restrictive view of what it is, or is not scholarship. 

Recently, scholarship has become synonymous with a number of other terms. 
Some of these include "research," "research publications," "publications," "faculty 
productivity, "scholarly works," and "publication productivity," to mention a few. 
According to Sundre (1990), the discussions surrounding the construct of faculty 
scholarship are numerous and remain largely uncoordinated, unspecified and void of 
empirical attention. Therefore, these lack definitional clarity and consensus. 
However, these definitions are still closely associated with research and publications. 
In other words, the main issue surrounding publishing is the limited way in which 
publications are defined and counted (Pellino ct al, 1984; Seldin, 1984; West et al, 
1980). These observations are shared by others in the field (Balch, 1980; Boyer, 
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1990; Forbes. 1977; Hays, 1989; Holt, 1988). This may be one of the reasons that 
O'Reilly (1989) asserts that the definition of scholarship has been misused in the 
process of making important formal decisions which affect faculty. 

Sundre (1990) reports that the main focus of the criticisms which surround the 
construct of scholarship is grounded in and centered on the narrowness that its 
operational definitions have taken on over the years. She calls for expanding and 
broadening the definition of faculty scholarship to include components of those 
activities, processes, and products beyond those traditionally assessed in the faculty 
evaluation process. This call is echoed by others (Boyer, 1990; Forbes, 1977; Miller 
& Serzan, 1984; O'Neill, 1987; Seldin, 1984). Sundre (1990, p. 4] believes that, "...as 
with most constructs, scholarship may be sufficiently complex as to defy a single 
definition; the challenge may be to provide variations on the definition for faculty 
members within different disciplines and fields." 

In a later study, Sundre (1992) explored, expanded, and clarified the content 
domain of the term scholarship. It was motivated by, and built upon contributions 
and suggestions from previous studies such as Braxton and Bayer (1986), Braxton and 
Toombs (1982), Creswell (1985, 1986), Pellino et al (1984), and Sundre (1990). The 
purpose of this research was to provide definitional clarity to the faculty scholarship 
construct by determining what faculty perceived as attributes of scholarship. 
According to Sundre (1990), this was the first time that faculty, rather than existent 
literature, were used to attempt to define scholarship. 



Using a qualitative approach, Sundre (1990) asked faculty members to 
generate components and attributes of scholarship. The results, obtained from full- 
time faculty from a doctoral-granting institution, was the development of an inventory 
of 249 attributes of faculty scholarship in the disciplines of theater, art, dance, and 
music. This inventory incorporated all the traditional role components of a faculty 
member as well as a large number of attributes not previously specified for these 
disciplines. 

Sundre (1990) states that her study provides evidence that the construct of 
faculty scholarship is more complex than previously recognized, and at a minimum, 
should no longer be used as a synonym for publication. Her findings apply to one 
regional doctoral-granting institution in the disciplines of theater, art, dance, and 
music. Therefore, they may not apply to other regional doctoral-granting institutions 
in these disciplines, or others. However, the listing of attributes developed as the 
result of this study provide an excellent starting point for- the specification of the 
definition of scholarship at other institutions with similar or different missions as well 
as other disciplines. 

Faculty Perceptions about Scholarship 
According to McShane and Douzenis (1987), the majority of faculty approves 
of research and publication as a method of evaluating scholarly contributions, but 
feels that too much emphasis is placed on publication. This conclusion is 
corroborated by others who assert that too much emphasis on publication takes 
faculty away from students, and that this adversely affects the quality of teaching 



Carnegie Foundation, 1989; Donald, 1985; Pellino et al, 1984; Schuster, 1990; 
Watkins, 1990). 

Fox (1992) conducted a survey of the attitudes that social science faculty have 
about scholarship, as measured by publication productivity, versus teaching relative 
to departmental reward structures. The findings of this study for the social sciences 
indicates that faculty members whose publication productivity is high are not strongly 
invested in both research and teaching. These results suggest that faculty members 
do not perceive that scholarship and teaching activities represent different aspects of 
a single dimension, but are different dimensions which are often at ^dds with each 
other. In other words, faculty perceive scholarly activities to be in competition with 
those of teaching. Comparable analyses are needed for faculty members in schools 
of education. 

Faculty in almost every institutional type perceive pressure to obtain external 
funding, conduct research and publish results (Blackburn et al, 1991; Lucas & 
Harrington, 1990; Pitz, 1992). The expectation to publish and its resulting pressure, 
place an unnecessary burden on faculty Seldin, 1984). This burden is 
counterproductive to quality publication output since creativity does not thrive when 
faced with deadlines and pressure to publish (McShane & Douzenis, 1987). 
Moreover, faculty are of the opinion that administrators rarely provide guidance on 
what is expected of publications and do not explain the criteria used to evaluate 
publications Sachs, 1988). They also feel that the criterion most often used to make 
formal decisions about their publications is the quantity of publications and that there 
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is little or no regard given to the quality or merit of the publication* (Carnegie 
Foundation, 1989; Carnegie Foundation, 1989; O'Reilly, 1989). In addition, they feel 
frustrated by the fact that they are supposed to conform to the conventional view that 
they are all scholars and publishers while they know they are not (Pellino et al, 1984). 
As a result, faculty members feel that they would be better teachers without the 
pressures of doing research and publishing (Cardozier, 1991). Despite these 
arguments, many still see that research and publishing are essential to improving the 
quality of teaching. However, there is only marginal evidence to support this 
rationale (Centra, 1989; Feldman, 1987; Fox, 1992; Wanner et al, 1980). 

Boice and Johnson (1984) reported that their study of the perceptions 
of 685 faculty members at a doctoral-granting university identified patterns and 
conditions under which faculty write, factors which inhibited writing habits, and 
techniques employed by faculty to facilitate the writing process. Their findings 
demonstrated the following: that faculty perceive that too much time and effort is 
devoted to the process of writing and publishing; that they used unsystematic methods 
to write; and that they felt the lack of time was the major obstacle to producing 
written work. 

Bieber and Blackburn (1989) state that faculty believe that doing a study today 
is more difficult and complex than it was yesterday. In order for research to be 
considered "quality" research, it now requires a large data base and the most 
sophisticated treatment of data. As such, faculty feel that it takes a longer period of 
time and a combination of expertise to produce an acceptable article. This is one of 
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the reasons cited by faculty for not increasing individual publication rates, and for 
granting a greater amount of credit for articles which are co-authored. It has also 
been reported that 83% of faculty are now saying that tenure was more difficult to 
obtain without evidence of extensive publication (Carnegie Foundation, 1989). This 
figure is up from 44% since 1969. 

Baird (1980) conducted a national study consisting of 511 chemistry, 598 
psychology, and 584 history faculty members of doctoral programs. This study 
explored the relationship between a department's scholarly productivity and its 
reputation within it field. It reported a substantial positive relationship between the 
two correlates, thereby promoting the idea that faculty perceive scholarly 
contributions and reputation as indicators of the quality of a department. 

Administrators' Perceptions about Scholarship 
Administrators approved of research and publication as a method of 
evaluating scholarly contributions made by faculty (McShane & Douzenis, 1987; 
Miller & Serzan, 1984; Seldin, 1984). While the majority of administrators have 
significantly higher expectations of faculty for scholarship and publication, those at 
research universities place more importance on research and publication as evidence 
of scholarly contributions than do those at liberal arts colleges (Pellino et al, 1984; 
Seldin, 1984). The importance placed on the role of research and publication is 
centered in the desire for securing, maintaining, or expanding the prestige of the 
university which results in a larger share of monies for a university Blake & Tjoumas, 
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1990; Watkins, 1990; West et al, 1980). In addition, it is felt that publication rates 
provide evidence of a general high energy output of faculty (Braxton & Toombs, 
1982). 

Because some administrators are of the opinion that there is too much 
superfluous publishing, they have adopted policies which limit the number of 
publications (Mooney, 1991). Stanford University and Harvard University Medical 
School are two examples of institutions which have adopted such policies. This is 
being done in an effort to emphasize quality rather than quantity and not equate the 
two. It is also being done to lessen the burden of peer reviewers, to enable faculty 
to spend more time with students, and to discourage the proliferation of narrowly 
specialized research. 

According to Suppa and Zirkel (1983), their study of 494 deans and 
chairpersons revealed the following information: 89% regarded articles in refereed 
journals as significant evidence for making promotion and tenure decisions; 72% of 
the respondents identified the writing of books and monographs as lesser scholarly 
activities; 7% felt that obtaining funding grants was perceived as important; 69% 
considered conference presentation significant; 53% gave more weight to non- 
refereed journals than to publications in regional or state journals which were 
refereed; and 54% felt tha. the importance of publications in refereed journals has 
grown in importance during the past five years. 

Creswell and Brown (1992) state that administrators arc in a position to 
facilitate and enhance the scholarly activities of their faculty members. This has been 
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echoed by others (O'Connell & wergin, 1982). Creswell and Brown's (1992) study 
consisted of interviews conducted with 33 academic department chairpersons 
representing a wide variety of disciplines, including education, at 15 research 
institutions. They identified seven categories and three unifying roles which 
administrators felt enhanced scholarly activities. Those were as follows: 1) 
administrative duties (provider, enabler); 2) externally oriented responsibilities 
(advocate); and 3) good interpersonal relations (mentor, encourager, collaborator, 
challenger). 

The results of this study are useful for identifying concepts of the role of 
administrators which correlate to improving faculty scholarship. However, as a 
model-developing study, there are three major limitations. First, the 33 cases used 
in the study were drawn primarily from doctoral-granting research universities and 
do not give a complete picture of other types of institutions. Second, the study was 
limited to the perspectives of chairpersons. Faculty perspectives are lacking and 
would be necessary for completeness since the use of chairs' self-reported 
perspectives may differ from actual behavior. Finally, neither gender or racial issues 
were addressed in the study. Despite these limitations, the work of Creswell and 
Brown (1992) is valuable as a tentative list of intervention strategies which can be 
expanded upon. 

Measures of Scholarship (Publication) 
Publishing is a word that bears a powerful message in academic circles. 
Research on this subject traces it roots to approximately 75 years ago, but little 
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attention was given to publishing until the 1960's (Bieber & Blackburn, 1989). Since 
that time, the professional societies and academe have persisted in pressuring faculty 
to publish promiscuously (Spaltro, 1980). 

The meaning of publishing and methods of evaluating it vary from institution 
to institution. Many universities, as part of the their evaluation processes and 
collective bargaining agreements (or facsimiles thereof), have established quotas for 
the number of articles expected to be published each year. In some cases only senior 
authorship or publication in refereed journals are counted as publications for 
evaluative purposes (Bracey, 1989). In other cases, administrators evaluate 
scholarship primarily on the quality of research publications in which the articles 
appear, while still others simply concentrate on the numbers of publications produced 
(McShane & Douzenis, 1987). 

Others question whether standards of evaluating faculty productivity should 
be the same across disciplines (Cardozier, 1991). Regardless, expectations and 
measures of scholarly publication are by nature qualitative and subjective and difficult 
to measure (Salomone & Vorhies, 1985; Seldin, 1984; West et al, 1980). Properly 
assessing faculty output is an important problem whose measurements hinge on 
accurately assessing the quality and quantity of knowledge that faculty produce 
(Bisber & Blackburn, 1989; West et al, 1980). However, before presenting the 
literature on measures of scholarship and the methods which should be used to 
evaluate publication, the central issue of faculty and administrator perceptions about 
scholarship will be presented and discussed. 
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Q uantitative and Qualitative Measures of Scholarship 

According to Seldin (1984), policies, practices, and measures currently used 
to evaluate faculty performance in the area of scholarship arc becoming more 
systematic. In addition, virtually all relevant information is considered in the 
evaluative process. Accurate app raisal of this, information hinges on measuring the 
quantity and quality of a faculty member's work (Centra, 1977; 1989). Therefore, 
measures of evaluating faculty publications as one measure of evaluating scholarly 
contributions can be largely grouped into quantitative and qualitative categories. It 
is for this reason that the literature has been examined from both of these 
perspectives. 

Quantitative Perspectives 

One of the most current practices which has emerged in faculty evaluation has 
been the use of publication counts (Seisin, 1984; Sundre, 1990; Walberg, Rasher, & 
Mantel, 1977). This is not to be confused with citation analysis which will be 
discussed in the "qualitative" section of this review. Publication counts and other 
methods of evaluating publications have been observed and studied by many over the 
past 20 to 30 years. This discussion will begin with those focusing on publication counts. 

Smith and Fiedler (1970) assert that measuring quantity of faculty 
performance in terms of publication counts has limitations and cite that a poorly 
conceived and written article published in a badly edited journal may count as much 
as a major contribution to the field which is well done and published in a refereed 
journal. They go on to state that publication norms and counts differ, or should 
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differ, from field to field. For example, articles in chemical journals are short and 
more easily written than those in fields such as philosophy. 

Methods of evaluating the scholarly work of faculty are beginning to appear 
on the international front. The results of a study of faculties of Canadian schools of 
education conducted by Arlin (1978) imply that the use of publication counts should 
be only one aspect of evaluating scholarly work. He cautions against the solitary use 
of publication counts and advocates some measure of quality which encompasses 
more than a quantitative focus. Evaluation of scholarly work through the use of 
quantitative methods only may be deleterious to the field of education by contributing 
to the proliferation of articles which are of questionable value (Arlin, 1978, O'Reilly, 
1989). 

Also on the international front, is the discussion by West, Hore, and Boon 
(1980) regarding the development of quantitative measures of evaluating the work of 
faculty members of Australia's institutions of higher education. These authors 
question reliance on quantitative measures despite their feelings that administrators 
seem to favor using quantitative measures in faculty evaluation procedures. 

According to West et al, (1980), the use of publication counts is associated 
with problems. One problem they identified is the effect that faculty personality has 
on publication rates. Some faculty are quick to publish, while others take more time. 
Another difficulty with using publication rates is that a single paper may contribute 
to more than one hundred others. 
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Seldin (1984), contends that the use of publication counts as an appraisal of 
scholarship is hazardous. He cites the example of some professors who write ten 
articles, while others write the same article ten times. According to Seldin, counting 
only referced journals as the only bona fide source of publication counts also falls 
short in evaluating scholarship since there are many excellent journals which are not 
refereed and many other forms of scholarly work beyond publication. He also states 
that refereed journals are subject to editorial needs of the moment, varying standards, 
and human fallibility. 

Bieber and Blackburn (1989) studied faculty research productivity over time 
as measured by publication counts. This study differed from others because it took 
into account the fact that both the amount of available journal space and the number 
of faculty competing for publication had changed over time. Their results 
demonstrated that the average available space in journals per faculty member has 
essentially doubled since 1972, thereby decreasing the level of competition among 
faculty to publish. Their study also showed that more, rather than fewer, faculty 
members are publishing than ever before. However, the study found that faculty 
effort to publish needs to almost double the publishing efforts of their 1972 
counterparts since, in 1972 units, an article published in 1988 is only about half of 
what it was in 1972. This has resulted in an improvement in quality since articles 
published in 1972 would, more often than not, not be accepted today. 

In addition to publication counts, there are other issues in the literature which 
arc germane to discussions about quantitative methods cf evaluating publications. 
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The information which follows is being included in this review in on>r to place the 
merits of the preceding material into perspective. 

Ladd and Lipset (1979) and Berelson (1960) report that publication is 
performed by relatively few faculty members with almost 90% of the articles being 
written by about 10% of college and university faculty. More recently, Boice, Jones 
(1984) and Ladd (1979) found that nearly three-fifths of full-time faculty have never 
published any sort of book, while only approximately one-fourth have published 
extensively (defined as 20 or more articles and/or three monographs). 

Blunt (1976) stated that the actual number of publications needed to achieve 
tenure and promotion is not large. For promotion to the rank of associate professor 
from that of assistant professor, he stated that an average of tluje refereed articles 
is required, while an average of five is required for promotion to the rank of full 
professor. 

Baird (1980) studied the relationship between ratings of graduate departments 
and faculty publication rates. He found that the utility of the ratings as an indicator 
of the scholarly contributions of single departments is questionable. In addition, he 
found that any particular publication measure should not be used as an overall 
measure of a department's "quality." 

As a result of their study, Wanner and others (1980) found that a unitary 
model of scholarly productivity cannot be assumed to operate equally in all academic 
disciplines. They demonstrated that mechanisms used to determine publication 
counts are different among disciplines. Their data showed that, independent of 
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publication counts, the process of determim.ig scholarly output depends on the 
intellectual context of each discipline. One issue this study raises is the influence of 
social structure on publication and productivity. For example, what constitutes an 
article and publication varies across disciplines. 

Wilson and Mandell (1981) proposed a model by which professior .1 activities, 
one of which is publication, may be quantified. The model consists of possible 
professional activities, each given a weighted point value. Baseline points gave 
parameters upon which all faculty were expected to be evaluated, and merit points 
were calculated for each of these activities. The resulting numbers were then used 
to reward faculty performance. Without subjecting the model to any study, its effect 
and validity are questionable. However, it does have value by serving as a catalyst 
for suggesting other areas of continued study, or approaches to faculty evaluation. 

Miller and Serzan (1987) report that little is known about the criteria of a 
refereed journal or the distribution of these journals across disciplines. They claim 
that the research in this area which separate refereed from non-refereed journals has 
not defined the quality or characteristics necessary to make the distinction between 
the two types of journals. However, despite this testimony, they cite evidence which 
demonstrates that members of promotion and tenure committees often do not read 
a candidate's publications, but judge the quality of an article by the reputation of the 
journal in which it appears. Moreover, according to Seldin (1984), referees have 
been known to reject manuscripts because of differences in philosophical, rather than 
scholarly, standpoints. 
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DeYoung (1985) contends that terms such as "production" and "output" are 
industrial concepts and should not be used in conjunction with the faculty evaluation 
process. He cautions against the use of such terms and posits that their continued 
use underscores the penetration of the technologic model into academe. 

Contrary to figures cited earlier in this section, Mooney (1991) reports that 
according to unpublished data from the federal government's Department of 
Education, the proportion of faculty at doctoral and research institutions who have 
published refereed articles in 1986 and 1987 was 70%. Counter to the figures cited 
by Mooney (1991), Cardozier (1991) reports that no one has documented substantial 
changes in faculty research and publication since the Ladd and Lipset study of 1979, 
and he does not feel that there is reason to believe that the information obtained 
from that study has changed much since that time. 
Qualitativ e Perspectives 

One of the most major methods which has emerged as the state of the art 
analysis of assessing scholarly work is citation analysis (Centra, 1989; DeYoung, 1985; 
Seldin, 1984). This methodology, made possible by the creation of the Social Science 
Citation Index (SSCI) in 1966, uses information on the number of times an article 
or book is cited by others in a given year. The SSCI indexes over 2,500 journals and 
is used to count citations as a measure of assessing faculty scholarship [31, 35]. 
Studies which have examined the use and validity of this measure of assessing the 
quality of scholarship follow. 
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Meyers (1970) examined the validity of using citation counts as a measure of 
productivity in the field of psychology. The results of his study, which spanned works 
published in this field between 1962 to 1967, found citation counts to be a valid 
measure of judging the quality of scholarly work. These results were confirmed by 
a similar study done by Walberg, Rasher, and Mantel (1977). 

Smith and Fiedler (1970) were among the few researchers who mentioned 
advantages to the use of citation analysis as a measure of evaluating publications. 
These include: citation index is not greatly influenced by quantity thereby 
emphasizing a publication's qualitative aspects; quantitative aspects can be eliminated 
from this measure by simply dividing the number of citations by the number of 
publications; the index of citations is relatively easy to obtain if the field has been 
referenced by the SSCI; citation indexes are based upon evaluations of research as 
opposed to personal evaluations; and, finally, citation by colleagues can be viewed as 
equivalent to their having a voice in the outcome of evaluating the merit of one's 
scholarly work. 

Despite the advantages cited by Smith and Fiedler (1970), they also reported 
limitations. First, a work may not be recognized for a considerable period of time. 
Second, a work may become so famous that it becomes public domain and is no 
longer cited by name. An example of this is the Student's t. Third, differences in 
fields needs to be considered. Heavy competition for citation may exist in some 
fields, such as chemistry, but not in others. Finally, there may be inherent biases and 
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an "old boy network" at work because of the fact that a researcher has a choice in the 
sources cited to support their particular work. 

Kroc (1984) studied the use of citation analysis on 51 schools of education. 
Although he found the SSCI useful and a valuable source of information, some 
practical problems and troublesome issues were identified with relying on this 
methodology as tLe major source of evaluation scholarship. Some of these include: 
the confusion created when two authors in a field have similar names, introducing a 
source of error; concerns raised by the practice of self-citation used by some scholars; 
criticism that the use of citation analysis is biased towards the logical positivism and 
technological paradigms of hard science; effects which shifts in research interest have 
within a field; the fact that SSCI only references the first author's name in the case 
of citing multiple authors; and the fact that many quality journals are not included 
in those referenced by the SSCI. 

Menges (1984), felt that the use of citation analysis was one, and only one 
method of measuring the value of scholarship. He posits that it is one way of 
assessing the impact that a scholarly work has in a field by providing information on 
how other scholars have reviewed and use a scholarly work. Miller [44] agrees with 
this assessment, however advises that caution be exercised when heavily relying on 
citation analysis as the major source of evaluation scholarly works. Menges 141] does 
note that the use of nonprint evidence of scholarship should be considered in 
evaluating scholarship, and these are not referenced in the SSCI. This is a 
disadvantage of this method of evaluation. 
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De Young (1985) feels thai citation analysis, as a sophisticated technique for 
evaluating publications ignores difficult questions raised by the use of such measures. 
Some of the pitfalls inherent in this technique consist of: bias against scholarship not 
found in officially published forms Oournals or books); failure to consider that some 
educational subspecialties, such as art education and philosophy, are less entrenched 
in the technological myth than are others; and overly optimistic notions of the 
scientific nature of education research would be applied to education practice. He 
asserts that any one methodology, regardless of its simplistic appeal, cannot stand in 
the place of understanding the complex world of academic scholarship and should 
only serve as a tool in its pursuit. 

Centra (1989) reports that citation analysis is increasingly being used by 
personnel committees, particularly those charged with the responsibility of making 
decisions about promotion and tenure. Centra identifies three problems with citation 
analysis as a measure of performance. First, there are differences in citations by 
discipline. Second, citations may be critical rather than positive. Third, there is the 
possibility hat the significance of work produced may not be recognized by 
contemporaries. 

Baird (1986) examined the correlates of average departmental scholarly 
activities in the form of publication rates in three disciplines: chemistry, history, and 
psychology. The national sample of this study consisted of 51 1 chemistry, 584 history, 
and 598 psychology faculty members from 24 chemistry departments, 25 history 
departments, and 25 psychology departments. The results of the study identified 
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three common correlates which enhanced the quality of scholarly work: 1) the 
presence of well trained faculty; 2) able students; and 3) support for scholarly 
activities. Group morale, humaneness of the environment, and collegiality were not 
found to be related to the quality or quantity of scholarly activities. 

One of the key result of Baird's study was the pattern of variables faculty 
members associated with scholarly productivity varied by discipline. This variation 
was rooted in the discipline's tradition and operating procedures. Knowledge of 
these discipline-related patterns may increase an understanding of the factors which 
promote scholarship in other disciplines. 

Nelson, Buss, and Katzko (1983) studied other procedures for assessing the 
quality of scholarly publications. They identified and discussed using direct and 
indirect evidence for evaluating publications. Direct evidence include peer reviews 
of articles or books as well as citation counts while indirect measures are those which 
apply to the rating of the journals publishing the articles. 

In addition to the problems of using citation analysis previously identified by 
others, Nelson et al (1983) also feel that the use of this citation analysis as a direct 
methodology is difficult to justify because of other technical problems which arise. 
Some of these include: differences in citation habits between disciplines making 
cross-disciplinary comparisons difficult; multiple authorship in some disciplines is 
rotated or assigned alphabetically making it difficult to index research done by an 
individual under other names; and length of time it takes to derive citation index 
because this methodology requires citation by others in subsequent publications 
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making citation index not suitable for use at salary and promotion hearings where the 

previous year's publications are an issue. 

Emerging Trends in Recommendations for 
Expanding the Definition of Scholarship 

In a recent survey of academic vice presidents and deans at more than eight 

hundred colleges and universities, Miller (1984) discovered that there was 

overwhelming support for viewing scholarship as more than research. Others have 

found that faculty agree with the association of research and publication [39, 40], but 

they are of the opinion that the scope and definition of scholarship and publication 

should be expanded and be more inclusive (Boyer, 1990; Mayer, 1984; Seldin, 1984; 

Sundre, 1990). This growing body of evidence supports the creation of new 

definitions for these terms. However, before a new definition can be developed, 

some reflections on how scholarship could be expanded need to be considered. The 

following authors provide food for thought foi framing and shaping the construct of 

scholarship and publication. 

Forbes (1977) defines scholarship as the act of studying and learning and, as 

such, states that it has little, or nothing to do with publication. He posits that the 

traditional definition of scholarship has inherent biases which exclude the perspectives 

of Native American, Chicano, black, and other non-European perspectives. 

Forbes (1977) accepts that one aspect of scholarship is publication. However, 

he questions the value of limiting scholarship to publication producing works to which 

only other scholars arc exposed. He concludes that this limitation breeds an 

incestuous relationship between professors which corrupts knowledge by perpetuating 
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ethnic, cultural, gender biases, and is the highest form of academic elitism. His 
opinions are captured by T.S. Eliot (1939, pg. 38] who said: "...we write for our 
friends, most of whom are writers, or for our pupils, most of whom are going to be 
writers; or we aim at a hypothetical popular audience which we do not know and 
which perhaps does not exist. The result, in any case, is apt to be a refined provincial 
crudity." 

Seldin (1984) calls not only for a redefinition of the term scholarship, but also 
suggests embellishing the meaning of the term publication. He believes that paper- 
based publication is becoming anachronistic and, as such, alternative methods of 
publishing, like computers, are growing in importance as vehicles to disseminate 
knowledge. It is for this reason that he recommends that convention of form, length, 
and access be included in an expanded definition of scholarship. 

In the study done by Pellino et al, (1984), six dimensions of scholarship 
emerged from their survey of approximately 1,000 faculty members and 55 
administrator in 24 colleges and universities. These included: scholarship as a 
professional activity; scholarship as research/publication; scholarship as artistic 
endeavor; scholarship as community service; scholarship as pedagogy; and scholarship 
engagement with the novel. 

Scholarship as a professional activity was characterized by a faculty's sense of 
responsibility for maintaining standards of quality through the processes of critique 
and review and through such activities as presentations and obtaining research 
funding. Scholarship as research/publication was associated with the commitment to 
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knowledge production and dissemination through traditional scholarly channels. 
Scholarship ao artistic endeavor implied an orientation and involvement with the arts 
as evidenced by performance and public aspects of artistic creation as well as 
enhancing one's own talents. Scholarship as community service was indicated by 
involvement in service-related activities external to the college or university and 
included the sharing of the faculty's expertise through consulting services or making 
presentations to civic or religious organizations. Scholarship as pedagogy clearly 
centered around activities involving the instructional process. Scholarship as 
engagement in the novel, considered a new conception, lacks the clarity of the 
previous five. However, what characterizes activities associated with this dimension 
is a continued emphasis on the new, or that which integrates one with another. For 
example, the introduction of scholarship in consulting could be viewed as a 
manifestation of scholarship as community service. 

According to DeYoung (1985) the discussions of scholarship and consistent 
with technological arguments in this area, is the assumption that whatever scholarship 
is, there must some tangible component of it which can be quantified. It is his 
contention that this assumption underscores the continued penetration and 
stronghold that the technological myth continues to have on institutions. The very 
concept of "scholarly productivity" confuses efficiency and rationality with reflection 
and creativity. His definition of scholarship is less linked to the scientific model and 
includes creative efforts which contribute to the discovery of new information, the 
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generation of new hypotheses, the publication of improved theoretical insights for 
explaining individual and/or philosophical relationships. 

Miller (1984) believes that scholarship is a state of mind which includes the 
following dimensions: content mastery, an inquiring attitude, rigorous examination of 
all evidence, accuracy in the use and interpretation of data, a willingness to discard 
the old for the new, and productivity. He feels that these dimensions should be 
present in three specific aspects of scholarship presentations, publication, and 
research. Presentations include speeches, panel participation, and workshops. He 
cautions that publication, as an element of scholarship which is generally subsumed 
under research, should be more than counting citations, or crediting those only 
appearing in rcfereed journals. Research according to Miller, satisfies curiosity, 
solves problems, proves or tests hypotheses, and enhances professional growth. As 
such, avenues for communicating research results need to be opened, and sometimes 
this may, by nature and circumstance, be outside of what has been traditionally 
defined and accepted as "scholarly journals." 

Hays (1989) agrees with the traditional "tangible and quantifiable" aspects of 
a definition of scholarship. However, she feel that this element separates artistic 
creativity and traditional scholarship, and advocates incorporating artistic and creative 
equivalents into a definition of scholarship. In order for this to happen, she believes 
that academic institutions must first accept that artistic and creative efforts arc 
equivalent to, and worthy of, scholarship. To help effect this change in thinking, she 
recommends a definition which includes bringing about something new and valuable, 
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the combining or organizing of existing materials into a new form, or any product of 
the power of scientific, artistic, or practical construction. 

Thomas (1989) defines scholarship as the essence of the teaching-learning 
process— the pursuit of knowledge. The author feels that unless broad definitions of 
scholarship are accepted, there is a temptation to become too narrow in focus 
resulting a loss of quality which naturally comes with diversity of thought and 
expression. He argues that academe has outgrown its traditional symbols and images, 
and calls for a more accurate and contemporary image of educational institutions 
which portray scholarship in a broader, diverse, more inclusive context. 

Boyer (1990) advocates scholarship which would be inclusive of four 
overlapping, but separate functions: the scholarship of discovery, the scholarship of 
integration, the scholarship of application, and the scholarship of teaching. The first 
type of scholarship, that of discovery, is closely akin to research. It contributes to 
human knowledge, confronts the unknown, and seeks understanding for its own sake. 

The scholarship of integration gives meaning to isolated facts and puts them 
into perspective, such as the case might be when making connections between and 
across disciplines. This type of scholarship enables specialties to be placed in larger 
contexts and bring new insights to bear on original research. 

The scholarship of application involves scholarly service through shaping public 
policy, working with public schools, etc. Finally, the scholarship of teaching is the 
bringing of the most honest and intelligible account of new knowledge to all who try 
to learn. 

n j" . 
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Summary 

From the results of the literature review, it is evident that much attention has 
been paid to the study of then construct of scholarship as part of faculty evaluation. 
The literature acknowledges and re-affirms the use of the traditional model of 
research, teaching, and service for the purpose of evaluating the performance of 
faculty across disciplines and institutions. However, the literature suggests that the 
ordering of the elements of the model differs, or need to differ, by discipline, 
in citutional type, and institutional purpose. 

In doctoral-granting institutions, especially in the arts and sciences, scholarship 
is equated with research as measured by publication, and is the most important 
criterion upon which faculty evaluations are based. Teaching is the second important 
criterion followed by servxe. There are no studies in the literature which delineate 
the variety of perspectives that exist among faculty in schools of education with 
varying levels of experience and productivity about the ordering of the model's 
criteria. 

The literature currently defines scholarship as research and publication, the 
constructs of which are complex and lack clarity. According to the literature, faculty 
generally agree with the use of the traditional model of research, teaching, and 
service as part of the evaluative process. However, in the social sciences, faculty feel 
that too much emphasis is placed on research and that there is undue pressure to 
publish. There are no studies about the perceptions of faculty in schools of 
education on this subject. There are also no studies in education which consider 
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variables such as the level of experience and productivity on faculty perceptions of 
scholarship. 

The literature reveals that administrators in doctoral -granting institutions also 
agree with the use of the traditional model they rank research and publication as the 
most important criteria used in faculty evaluation. Generally speaking, research and 
publication are seen not only as measures of faculty productivity for the purpose of 
evaluation, but are also viewed as ways to enhance an institution's prestige. 

Measures of evaluating scholarship vary from institution to institution. There 
are no precise indicators of these measures or methods, nor are there practices which 
codify these measures. However, the measures commonly identified have been 
grouped into quantitative and qualitative categories. The primary quantitative 
measure is that of publication counts. This method does not operate equally across 
disciplines or institutions. The literature is lacking in studies which delineate the 
varying of perspectives of faculty in a school of education with varying levels of 
experience and productivity about the use of this method, or the manner in which it 
is codified in schools of education. 

A common qualitative measure is that of citation analysis. Once again, 
according to the literature, this method does not account for differences in 
institutions or disciplines. There is little information about the perspectives of faculty 
in a school of education with varying levels of experience and productivity about the 
use of this method in evaluating their performance. 
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Generally speaking, there is overwhelming evidence in the literature which 
states that the current definition of scholarship is too narrow, lacks clarity, and needs 
expansion. This evidence is representative of a variety of institutions and disciplines, 
including education. However, there are no studies which have delineated the variety 
of perspectives of faculty in a school of education with varying levels of experience 
and productivity which have defined scholarship. 

In conclusion, the criterion of scholarship identified through the literature fall 
into five major categories: 1) definitions of scholarship; 2) faculty perceptions about 
scholarship; 3) administrators' perceptions about scholarship; 4) measures and 
indicators of scholarship; and 4) emerging trends and recommendations about 
scholarship. These categories appear to differ across and within institutions and 
disciplines. However, the information obtained from this review should benefit 
schools of education that are re-examining their faculty evaluation policies and 
practices. At a minimum, it may expand the term scholarship used in schools of 
education so that it will no longer be used synonymously with the terms of research 
and publication. 
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